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Brainwave  to  Computer  Interface  Development 


Richard  H.  Dickhaut,  Ph.D. 
Spectra  Research  Institute 
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Abstract 

Electroencephalographic  (EEG)  scalp  sensors  were  used  as  probes  to  acquire 
neuroelectric  data  correlated  in  real  time  with  cognitive  processing  of  linguistic  thought 
stimuli.  The  adoption  of  a  suitable  data  analysis  methodology  provides  results  that  show  a 
direct  relationship  with  a  characteristic,  quantifiable  set  of  neural  activity  for  digits  from 
zero  to  nine,  or  the  words  ‘yes’  or  ‘no’. 

An  experimental  design  was  centered  on  timed  capture  of  single  epoch  neuroelectric  data 
with  EEG  sensors  upon  the  initiation  of  thought  stimuli.  Experimental  results  were 
analyzed  for  three  test  subjects  and  twelve  linguistic  stimuli  at  two  electrode  locations. 
The  waveform  from  each  location  and  each  single  stimulus  event  was  subjected  to  a 
decomposition  procedure  based  on  the  premise  that  macrocolumns,  acting  as  modular 
processing  units,  contribute  their  own  simultaneous  subwaveforms  to  a  resultant 
composite  waveform  detected  by  the  EEG  sensor.  The  decomposition  subwaveforms 
analytically  obtained  are  the  result  of  an  exact  solution  of  a  set  of  equations  applied  to  the 
experimental  data  for  each  individual  epoch.  Each  decomposition  subwaveform  has  its 
own  distinct  frequency,  amplitude,  phase,  and  decay  characteristic.  The  analytical 
outcome  yields  parametric  coefficients  that  determine  the  characteristics  of  each 
sub  waveform. 

The  resultant  set  of  parametric  coefficients  from  a  group  of  thirty  single  data  epoch 
waveforms  were  used  to  generate  a  three  dimensional  parametric  surface  that  is 
characteristic  and  specific  for  an  individual  digit  or  word  at  a  given  electrode  location  for 
each  test  subject.  Two  electrode  locations  were  chosen  that  are  associated  with  cognitive, 
linguistic  processing.  Each  digit  or  word,  then,  has  a  set  of  3-D  neural  activity  maps 
corresponding  to  the  two  electrode  locations. 


1 


SPECTRA  RESEARCH  Institute 


Background 

How  information  is  processed,  represented,  and  actively  used  in  the  brain  is  a  major 
question.  When  the  eyes  observe  a  written  word,  neural  spike  train  sequences  are  set  into 
play  in  nerves  everywhere  along  the  pathway  from  the  eyes  to  layers  of  the  neocortex. 
From  there  on,  immense  complexities  challenge  our  capability  to  model  the 
computational  behavior  of  large  groups  of  neurons  densely  interconnected  with  many 
thousands  of  synapses  per  neuron,  and  any  emergence  of  large  scale,  complex  system 
behavior.  The  notion  that  computation  is  involved  is  attractive  because  it  is  a  reasonable 
hypothesis  to  assume  the  brain  must  somehow  represent  information  for  short  time 
purposes  as  well  as  store  it  for  future  use,  and  electrical  representation  of  information  in  a 
compact  form  achieved  by  computation  is  a  familiar  concept. 

The  general  structural  characteristics  of  the  neocortex  can  supply  the  inspiration  and  a 
good  basis  for  model  construction.  Neocortical  neurons  reside  in  vertical  columns 
perpendicular  to  the  cortical  surface  (Mountcastle,  1957,1978;  Hubei  and  Wiesel,  1977; 
Nauta,  1979;  Kaas  et  al.,  1981;  Gazzaniga,  1989),  as  well  as  in  six  horizontal  layers. 
These  columns  have  been  referred  to  as  macrocolumns,  and  two  decades  ago  the 
suggestion  was  made  that  they  might  be  modular  processing  units  (Rakic,  1976). 

Two  1996  papers  supply  useful  insights  that  could  support  the  idea  of  macrocolumns  as 
modular  processing  units.  Traub  et  al.  (1996)  describe  intemeuron  networks  that  generate 
a  distribution  of  20  -  70  Hz  oscillations  arising  from  spike  doublets  dependent  on 
intracellular  current  injection.  Intragroup  connections  between  networks  result  in 
coherent  oscillations.  The  paper  by  Gray  and  McCormack  (1996)  introduces  a  subset  of 
pyramidal  cells  in  layers  2  and  3  of  the  visual  cortex,  called  chattering  cells  (CH),  whose 
stimulation  from  driving  current  injection  results  in  doublet  bursts.  These  experimentally 
observed  20  -  70  Hz  oscillations,  again  in  a  distribution,  result  in  synchronous  firing 
between  cells  in  different  macrocolumns.  The  CH  cells  also  receive  oscillatory  bursts  in 
the  same  range  from  intemeuron  sources.  The  doublet  bursts  are  an  effective  form  of 
presynaptic  input  —  rapid  firing  can  lead  to  postsynaptic  temporal  summation  and 
increase  dramatically  the  probability  of  presynaptic  neuron  transmitter  release.  CH  cell 
projections  provide  a  basis  for  generation  of  long  range  synchronous  cortical  activity. 

The  end  surface  area  of  the  columns  is  small  compared  to  the  surface  area  of  a  probe  of 
neuroelectric  brain  activity  such  as  an  EEG  electrode.  Suppose  that  the  information 
output  of  each  of  these  columns  is  more  or  less  independent,  but  some  redundancy  is 
allowed  among  groups  of  closely  associated  columns  that  may  be  computing  similar 
information  content.  Let  the  dynamic  change  in  electrical  charge  per  unit  time  in  these 
groups  of  columns  be  sensed  by  the  EEG  probe  over  a  cortical  gyrus,  and  let  there  be 
minimal  influence  from  weaker  sources  farther  away.  The  result  should  be  an  EEG  probe 
signal  representing  a  composite  electrical  signature  that  reflects  the  summation  of  the 
signatures  from  each  of  the  column  groups  more  or  less  directly  underneath  the  probe.  In 
the  case  of  the  cognitive  stimuli  experiments  being  considered  here,  the  signal  response 
will  be  referred  to  as  an  event-related-potential  (ERP). 
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For  modeling  purposes,  a  key  ingredient  is  the  selection  of  an  appropriate  analytical 
technique  that  can  operate  on  the  composite  ERP  signal  and  decompose  the  signature  into 
its  original  set  of  separate  waveforms.  The  technique  that  has  been  chosen  and  found 
sound  in  previously  published  research  (Dickhaut,  1988)  also  had  two  other  attributes  that 
were  added  to  the  cognitive  model  considerations  noted  above,  namely,  that  the 
experimental  system  output  was  produced  by  an  impulse  function  response  and  that  only 
pulse  waveform  phenomena  would  be  extracted  in  the  analysis.  The  first  of  these  two 
attributes  means  that  the  brain,  as  the  system  in  question,  would  be  considered  to  be  in  an 
ambient  ready  state  prior  to  the  sudden  appearance  of  a  stimulus  or  the  production  of  a 
thought,  and  would  then  have  the  capability  to  respond,  as  many  physical  systems  would, 
to  a  single  exciting  stimulus  (e.g.,  with  peak  amplitude  and  exponential  decay 
phenomena,  etc.).  In  the  application  of  this  idea  to  the  brain,  it  is  assumed  likely  that  the 
stimulus  response  is  subserved  by  real  time  dynamics  of  a  mechanism  akin  to  reentrant 
signaling  and  information  processing  as  proposed  by  Edelman  (1979).  The  second 
attribute  means  that  all  continuously  valued  wave  phenomena  with  the  same  frequency 
and  amplitude  throughout  the  duration  of  the  electrical  signal  would  be  ignored. 

The  particular  waveform  decomposition  technique  chosen  presumes  that  the  target  ERP 
signature  is  the  result  of  an  excited  structure  responding  to  impulse  phenomena,  and  is 
composed  of  multiple  simultaneous  waveforms  each  having  its  own  structural  frequency 
characteristic. 

Results 

Experimental  results  were  analyzed  for  three  test  subjects  and  twelve  linguistic  stimuli  at 
two  electrode  locations  that  lie  directly  over  cortical  areas  associated  with  cognitive 
processing.  The  two  areas  were  Brodmann’s  39/40  over  the  left  hemisphere;  and  02  for 
the  right  hemisphere.  Waveforms  were  recorded  for  the  resultant  neural  activity 
associated  with  the  thought  of  a  digit  ranging  from  one  to  nine  or  the  thought  of  the 
words  ‘yes’  or  ‘no’,  and  subjected  to  a  special  decomposition  analysis.  For  secondary 
comparison  purposes,  waveforms  were  also  recorded  and  analyzed  for  the  experimental 
condition  in  which  the  brain  responded  to  the  presentation  on  a  computer  screen  of  the 
same  digits  and  words.  For  both  experimental  conditions,  the  waveform  from  each 
location  and  each  single  stimulus  event  was  subjected  to  a  decomposition  procedure 
based  on  a  model  of  an  excited  structure  responding  to  impulse  function  neural  driving 
currents  that,  in  turn,  were  initiated  by  the  sudden  onset  of  the  cognitively  meaningful 
stimulus.  A  set  of  subwaveforms  was  produced  by  the  decomposition  procedure.  Each 
subwaveform  in  the  decomposition  set  has  its  own  distinct  frequency,  amplitude,  phase, 
and  decay  characteristic. 

The  next  step  in  the  process  was  to  use  the  analytically-derived  parameter  coefficients 
from  the  subwaveforms  to  produce  a  3-D  topographical  map  from  the  data  for  each 
individual  event  (i.e.,  physiological  voltage  activity  in  the  defined  time  epoch)  and 
stimulus  type,  as  well  as  electrode  location,  and  human  subject.  The  3-D  map  comprises 
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Figure  lc.  Subject  3,  parietal,  thought  stimulus  — 
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ure  3a.  Subject  1,  occipital,  thought  stimulus 
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Figure  3b.  Subject  2,  occipital,  thought  stimulus  = 
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Figure  3c.  Subject  3,  occipital,  thought  stimulus  - 
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Figure  5b.  Subject  1,  parie 
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Figure  6a.  Subject  1,  occipital,  thought  stimulus  = 
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Figure  6b.  Subject  1,  occipital,  image  stimulus  = 
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Figure  7a.  Subject  2,  occipital,  thought  stimulus  = 
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Figure  7b.  Subject  2,  occipital,  image  stimulus  = 
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the  parametric  data  of  specific  neural  activity  for  given  responses  to  a  cognitive  stimulus 
and  relates  the  frequencies  of  the  signals  activated  during  cognition,  their  intensities 
(including  phase  information),  and  the  rate  at  which  they  decay.  The  analytical  data  from 
approximately  thirty  epochs  of  the  same  stimulus  type  were  sufficient  to  establish  a 
stable,  unique  map  for  each  cognitive  stimulus,  electrode  location,  and  test  subject. 

Figures  1  through  7  show  examples  of  3-D  surface  maps  described  above.  In  each  of  the 
figures  the  x-axis  is  the  frequency  in  Hertz,  the  y-axis  is  the  decay  coefficient,  and  the  z- 
axis  is  the  amplitude  (including  phase).  Figures  with  the  label  ‘parietal’  in  their  title 
correspond  to  the  information  obtained  from  scalp  electrodes  placed  over  Brodmann’s 
area  39/40,  left  hemisphere.  Similarly,  figures  with  the  label  ‘occipital’  in  their  title 
correspond  to  electrodes  over  Brodmann’s  area  02,  the  right  hemisphere  occipital 
placement. 

The  primary  purpose  and  result  of  the  experiments  described  here,  supported  by  the 
computational  analyses,  together  with  the  3-D  plots,  is  the  demonstration  that  individual 
characteristic  neural  activity  is  obtained  for  specific  thought  stimuli,  electrode  locations, 
and  different  human  test  subjects.  All  experiments  provided  characteristic  3-D  maps  for 
all  the  experimental  conditions.  They  are  stable  maps;  the  topological  surface  features  of 
each  map  do  not  change  with  increased  amounts  of  data  from  further  acquisition  of 
experimental  epochs  for  each  thought  stimulus.  It  should  be  remembered  that  these  maps 
are  the  result  of  a  multi-step  process  of  abstraction  based  on  a  specific  computational 
model  for  cognitive  processing  described  earlier.  Each  map  comprises  about  300  data 
points  (on  the  average)  representing  the  decomposition  parameters  for  the  subwaveforms 
for  each  of  the  approximately  30  epochs  acquired  of  the  neuroelectric  activity  associated 
with  a  given  stimulus.  Thus  each  map  is  an  abstract  representation  of  the  neural  activity 
involved  in  cognitive  processing  of  a  specific  thought  stimulus  at  a  specific  electrode 
location.  These  parametric  maps  will  form  the  sufficient  basis  set  of  information  that  will 
enable  a  direct  identification  scheme  for  eventual  real  time  brainwave  computer  control 
applications. 

As  a  matter  of  secondary  interest,  there  are  a  number  of  scientific  questions  that  can  be 
asked  regarding  neural  activity  in  cognitive  processing  with  the  type  of'  experiments 
described  here  and  for  future  experiments.  As  an  example,  one  question  immediately  and 
always  asked  in  discussing  the  neural  activity  maps  and  cognitive  processing  is  whether 
there  is  evidence  for  similar  neural  activity  between  people  thinking  of  the  same 
linguistic  items.  The  presumption  in  the  question  is  that  even  if  totally  independent 
people  have  different  specific  brain  circuitry  and  language  experience,  there  has  to  be 
some  fundamental  process/mechanism  that  enables  people,  in  principle,  to  communicate, 
and  perhaps  it  might  be  similar  neural  activity  for  the  concepts  and  words  used.  A  second 
question  has  to  do  with  the  possibility  of  similar  cognitive  activity  in  a  given  individual 
when  thinking  of  a  linguistic  item  versus  responding  to  an  image  of  it.  There  is  some 
suggestive  evidence  in  the  neural  activity  map  data  that  both  these  questions  could  be 
addressed  seriously  with  further  refinements  in  experimental  protocol,  while  still 
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addressing  the  primary  issue  of  employing  real  time  brain  neural  activity  during  cognition 
to  direct  computer  functions. 

Discussion 

The  results  represent  a  minimum  pilot  study  in  which  EEG  ERP  data  were  acquired  and 
analyzed  for  cognitive  processing  characteristics  related  to  thinking  of  the  numbers  zero 
to  nine  and  the  words  ‘yes’  and  ‘no’.  The  model  considerations  leading  to  the 
experiments  have  been  described  above.  Stable  and  characteristic  neural  activity  maps 
were  predicted  to  be  found,  and  were  found,  for  thought  stimuli  retrieved  from  memory, 
just  as  they  have  been  for  responses  to  linguistic  stimuli  presented  on  a  computer  monitor 
screen  (present  experiment,  and  earlier  ones  yet  to  be  reported).  These  results  comprise  a 
first  step  in  the  possibilities  for  developing  a  direct  brain  to  computer  interface  based  on 
real  time  neural  activation  related  to  cognitive  processing  of  specific  information.  The  3- 
D  maps  of  abstract  parametric  data  describe  a  range  of  neural  activity  associated  with 
specific  linguistic  items.  The  idea  is  that  when  such  a  map  is  stored  in  a  computer 
memory,  a  subsequent  single  epoch  of  neural  activity  associated  with  the  same  linguistic 
item  can  be  identified  as  belonging  to  that  same  map,  given  an  analysis  of  the  ERP.  The 
identification  of  the  map  can  then  point  to  the  ASCII  word  correlate  for  immediate  further 
use. 

The  individual  epoch  ERPs  for  memory  retrieval  items  were  just  as  robust  as  those 
responses  to  images  of  the  same  linguistic  items  on  a  computer  monitor.  Although  time 
zero  (t0)  was  known  precisely  for  the  image  stimuli  experiment,  t0  identification  for  the 
memory  retrieval  experiment  posed  a  problem.  Acquired  data  after  the  pixel  cross  lit  up 
as  a  time  marker  on  the  computer  monitor  failed  to  reveal  any  obvious  indication  of  the 
start  of  cognitive  processing.  More  work  will  have  to  be  done  on  this  problem  in  future 
experiments  as  no  good,  obvious,  and  direct  method  was  found  to  solve  the  problem  for  t0 
identification.  An  indirect  method  served  for  the  present  study.  Calculation  of  a  time 
average  of  the  data  for  all  the  thought  stimuli  events  for  any  given  stimulus  number  for  a 
given  test  subject  and  electrode  location,  followed  by  comparing  that  waveform  with  the 
time  average  waveform  for  the  events  produced  by  responding  to  an  image  of  the  same 
stimulus  number,  demonstrated  that  the  two  waveforms  matched  in  some  time  history 
features  for  the  region  between  100  and  200  milliseconds.  This  was  true  for  all  stimuli 
and  electrode  locations.  Since  the  image  response  to  was  known,  the  thought  stimulus  t0 
could  be  inferred  to  be  approximately  the  same,  given  the  time  history  matching  features. 
On  this  basis,  a  uniform  t0  starting  point  was  chosen  for  all  thought  stimuli  events. 

Intercomparisons  between  Figures  la,  lb,  and  lc  show  a  similar  topological  feature 
among  three  subjects  in  the  band  of  neural  activity  corresponding  to  values  of  the  decay 
coefficient  from  10  to  25.  (Note  that  in  these  and  subsequent  figures  the  y-scales  are  often 
different  in  different  plots  dependent  on  data  outcomes  from  different  experiments).  The 
plot  for  each  subject  for  the  thought  stimulus  ‘2’  is  characteristic  for  that  individual,  but 
there  is  a  similar  3-D  surface  feature  in  the  region  near  the  coordinates  for  30  Hz  and  a 
decay  coefficient  of  25  for  all  three  subjects.  The  feature  and  surrounding  topological 
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region  suggests  that  there  may  be  something  common  in  the  neurological  activity 
regarding  cognitive  processing  of  the  number  ‘2’  among  separate  individuals.  Again, 
there  are  suggestions  for  common  topological  features  in  Figures  2a  and  2b,  for  two 
individuals  in  the  instance  when  the  thought  retrieval  stimulus  is  ‘9’.  Both  Figures  1  and 
2  are  for  the  parietal  electrode  location.  Figures  3a,  3b,  and  3c  show  the  data  for  the 
thought  stimulus  ‘9’  at  the  occipital  electrode  location  for  the  three  subjects.  Again,  there 
are  hints  of  common  features  to  look  for  in  more  refined  experiments.  Remember  there  is 
no  visual  stimulus  of  the  number  ‘9’  involved,  but  the  visual  cortex  is  showing  the 
evidence  for  cognitive  processing  of  the  thought  stimulus. 

In  Figures  4a  and  4b,  there  are  suggestions  of  common  features  for  a  given  subject  for 
both  the  parietal  and  occipital  location  when  the  thought  stimulus  is  ‘7’.  This  was  a  rare 
occurrence  in  this  preliminary  experiment  for  different  thought  stimulus  numbers  and 
subjects,  but  has  been  observed  before  in  image  stimulus  experiments. 

Figures  5a  and  5b,  for  a  given  subject,  show  the  parietal  neural  activity  data  of  the 
thought  stimulus  experiment  versus  the  image  stimulus  experiment  when  the  stimulus 
number  is  ‘7’.  Figures  5c  and  5d  do  the  same  for  the  occipital  neural  activity  data.  Given 
any  reasonable  feature  correlations,  the  question  is  whether  cognitive  processing  of  a 
thought  stimulus  has  something  in  common  with  the  cognitive  processing  of  an  image 
stimulus,  or  alternatively,  whether  perhaps  cognitive  representation  in  memory  of  a 
linguistic  item  is  always  simultaneously  involved  as  a  major  factor  in  real  time  cognitive 
processing.  Figures  6,  7,  and  8  show  more  comparison  data  for  thought  and  image 
stimulus  experiments. 

The  analytic  approach  used  here  yields  frequencies  that  are  not  harmonics  and  describes 
only  the  pulse-coded  behavior  in  the  original  signal  while  ignoring  everything  else  as 
background  noise.  Each  analytic  result  is  an  exact  solution  of  the  specified  number  of 
simultaneous  equations  and  conjugate  pole  pairs  in  the  experimental  data.  As 
implemented,  the  exact  solution  is  required  to  meet  some  explicit  mathematical  criteria, 
and  when  this  is  completed,  the  output  lists  the  parameter  values  for  the  irregular  basis 
set.  The  irregular  basis  set  describes  each  of  the  simultaneous  decomposition  waveforms 
in  terms  of  their  distinct  frequency,  amplitude,  phase,  and  decay  characteristics. 

The  group  waveform  set  as  described  above  provides  a  powerful  and  quantitative  analytic 
approach  to  macroscopic  electrical  signaling  produced  by  the  brain  in  response  to 
cognitive  stimuli.  As  noted,  the  nature  of  the  ERP  signature  has  been  presumed  to  be 
entirely  different  in  structure  than  has  been  classically  described.  In  the  most  general 
sense,  ERP  signatures  have  previously  been  described  and  analyzed  as  if  they  were 
unitary  signals  in  which  sequential  brain  events  during  cognitive  processing  produced 
consequent  and  reproducible  time-specific  components.  The  present  work  does  not  make 
the  assumption  that  the  ERP  is  a  single  unitary  signal,  but  rather  that  it  is  a  composite 
signal  that  can  be  decomposed  into  a  finite  set  of  individual  waveforms.  These 
waveforms  and  their  parametric  coefficients,  then,  would  be  the  analytic  quantities,  rather 
than  the  classically  named  N  and  P  components,  that  an  investigator  would  track  in 
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cognitive  experiments  that  manipulate  physiological  or  psychological  variables  as  part  of 
the  experimental  design.  The  reconstruction  of  the  ERP  signature  from  the  irregular 
decomposition  basis  set  shows  that  N  and  P  components  (such  as  the  P3)  are  artifacts  of 
the  summation  process  involving  the  waveforms. 

In  a  few  instances,  there  was  a  virtual  repeat  of  time  domain  waveform  characteristics 
starting  at  around  350  milliseconds.  It  was  not  a  consistent  observation,  but  it  might 
become  more  common  with  improved  experiments.  All  processed  data  used  a  time 
window  corresponding  to  the  time  period  between  50  and  562  milliseconds  after  the 
designated  t0. 

In  future  experiments,  there  will  be  an  endeavor  to  improve  experimental  procedures  and 
computer  techniques  for  data  acquisition  in  thought  retrieval  experiments.  In  the  present 
experiment  the  subject’s  attention  was  engaged  in  the  time  between  the  recording  of  the 
previous  item  and  the  end  of  the  random  time  interval  between  2.5  to  3.5  seconds  before 
the  pixel  cross  lit  up  on  the  computer  monitor.  As  reported  by  the  test  subjects,  the 
attention  requirement  introduces  some  element  of  stress  and  fatigue  over  the  experimental 
session.  One  possibility  to  avoid  this  problem  may  be  to  have  the  subject  think 
sequentially  of  three  or  four  numbers  in  a  row,  then  report  them.  The  data  acquisition 
scheme  should  be  designed  clever  enough  to  pick  up  ambient  data  500  msec,  or  so  before 
the  thought  retrieval  of  the  first  number.  The  current  procedure  is  also  open  to  the 
criticism  that  the  subjects  may  have  rehearsed  the  items  to  be  thought  of  prior  to  the  pixel 
signal  appearance,  or  otherwise  engaged  unconsciously  other  simultaneous  cognitive 
processing  mechanisms,  given  the  instruction  to  be  prepared  to  think  of  an  item  from  the 
stimulus  set  when  the  pixel  signal  appeared.  The  new  approach  should  help  reduce 
possible  contamination  from  these  sources  and  be  more  like  free  flowing  thought 
production. 

Experimental  Design  and  Procedures 

The  principal  experiment  was  designed  to  capture  physiological  voltage  signals 
associated  with  neocortical  information  processing  as  memory  retrieval  of  digits  and 
words  took  place  in  a  sequence  determined  by  the  test  subject.  A  small  computer¬ 
generated  cross  formed  from  a  few  pixels  was  presented  in  the  center  of  a  computer 
monitor  screen  and  was  the  signal  for  the  test  subject  to  think  of  a  digit  from  zero  to  nine 
or  the  word  ‘yes’  or  ‘no’.  After  the  thought  was  completed,  it  was  reported  to  the  person 
handling  the  data  acquisition  who  then  entered  the  reported  item  into  the  computer.  In 
this  manner  the  sequence  and  the  number  of  items  in  each  of  the  twelve  categories  were 
archived  for  each  experiment.  The  subjects  were  instructed  to  be  prepared  to  think  of  a 
number  or  word  from  the  target  list  of  twelve  items  as  soon  as  the  pixel  cross  was  lit.  The 
instruction  was  intended  to  limit  the  time  period  in  which  it  would  be  necessary  to  search 
for  indications  of  the  initiation  of  cognitive  processing  (or  time  zero)  associated  with  the 
thought  activation. 
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The  secondary  experiment  was  designed  to  elicit  physiological  voltage  signals  associated 
with  neocortical  information  processing  in  response  to  displayed  digits  and  words  on  a 
computer  monitor  from  the  same  set  of  items  used  in  the  primary  experiment..  A  special 
computer-generated  presentation  was  prepared  in  which  randomly  ordered  and  randomly 
spaced,  yet  precisely  timed,  stimuli  appeared  to  the  experimental  subjects  on  a  computer 
monitor.  The  randomness  was  achieved  by  use  of  a  standard  mathematical  card  shuffling 
routine  adapted  for  the  purpose  of  this  experiment.  The  precise  timing  enabled  correlation 
with  the  beginning  of  the  target  physiological  signal.  Each  of  the  twelve  stimuli  appeared 
forty  times  The  stimuli  also  had  random  continuous  perturbations  of  plus  or  minus  a  half 
second  around  the  average  stimulus  separation  time  of  3  seconds.  The  test  subject  was 
allowed  to  temporarily  halt  the  session  for  a  break  whenever  it  was  requested. 


Six  electrodes  were  placed  on  each  subject  at  frontal,  parieto-temporal,  and  occipital 
locations  over  both  hemispheres,  corresponding  to  Brodmann's  areas  46  and  39/40,  plus 
occipital  placement  O  j  on  the  left  hemisphere,  and  mirror  image  placements  on  the  right 
hemisphere.  Each  electrode  measurement  on  a  given  hemisphere  was  referenced  to  a 
mastoid  process  electrode  location  on  that  side  of  the  head.  All  recordings  are  of  unipolar 
signals.  The  experimental  protocol  was  an  instance  in  which  the  subjects  were  informed 
and  trained  subjects,  not  naive  individuals.  Each  subject  was  fully  informed  about  the 
design  of  the  experiment,  its  purposes,  and  the  procedures  to  be  followed.  A  personal 
computer  (PC)  was  used  in  the  preparation  and  presentation  of  the  pixel  timing  signal  and 
for  the  image  stimuli.  The  subjects  spent  up  as  much  time  as  needed  with  a  test  computer 
file  that  familiarized  them  with  the  stimuli  images,  and  provided  an  opportunity  to 
receive  training  for  the  short  duration  interval  (600  msec.)  of  each  image  stimulus 
presentation. 

A  computer-controlled,  multichannel  Grass  EEG  test  instrument  was  used  to  take  the 
EEG  data,  that  were  then  digitized  at  a  1  KHz  clock  rate  by  a  Keithley  Instruments  analog 
to  digital  (A/D)  converter,  sent  to  the  PC,  and  finally  stored  on  a  hard  drive.  Over  1.1 
seconds  of  data  were  recorded  in  this  fashion  for  each  of  the  six  data  channels  at  the  onset 
of  each  thought,  or  stimulus  presentation,  and  are  permanently  archived. 

The  data  analysis  simultaneously  extracted  the  frequency,  phase,  and  time-domain 
information  content  from  the  ERP  signature  and  presented  the  output  in  the  form  of  a 
basis  set  of  coefficients  that  describe  the  decomposition  of  the  signature  into  constituent 
subwaveforms.  Each  of  these  sub  waveforms  is  a  complex  exponential  function  with  a 
distinct  amplitude  coefficient,  unique  frequency,  phase,  and  decay  coefficient.  Before 
analysis  was  undertaken,  each  waveform  acquired  in  the  experiment  was  examined  for 
artifacts  disruptive  to  analytical  processing,  such  as  occur  in  muscle  movement,  stress, 
eye  blinks,  and  electrical  anomalies.  Waveforms  with  artifacts  that  would  preclude 
successful  processing  were  removed  from  the  data  set. 
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Waveform  Decomposition  Analysis 

The  basic  assumptions  regarding  the  manifestation  of  the  physiological  voltage  signal  are 
as  given  earlier.  Additionally,  however,  the  parametric  method  now  being  described 
assumes  that  the  signal  contains  pulse-coded  behavior  due  to  the  brain's  response  to  a 
semantic  stimulus  presented  during  a  special  experimental  procedure.  Under  these 
conditions,  the  brain's  response  has  been  shown  to  reflect  unique  information  generating 
events,  and  the  analysis  method  shown  to  decompose  successfully  the  composite  signal 
and  provide  a  description  of  the  underlying  pulse-coded  subwaveforms  (Dickhaut,  1988). 

The  response  of  an  electrically  active  neural  structure  to  an  impulse  function  as  described 
in  the  previous  paragraph  resulted  in  a  finite  sum  of  complex  exponentials,  for  which  a 
technique  known  as  the  SingularityExpansion  Method  (Baum,  1971;  Tesche,  1972)  can 
be  applied.  Similarly,  for  N  uniformly  spaced  time  samples  in  the  signal,  it  is  proposed 
that  the  response  of  the  structure  can  be  written  as 


/(/,.)  =  YJAJea/l  (1) 

j= i 

where  A=  complex  amplitude, 

a  =  complex  natural  frequency, 

n  =  number  of  independent  exponentials  in  the  data,  and 
/,=  (i-l)At. 

It  should  be  noted  that  the  complex  natural  frequency  does  not  contain  the  term  2  n  and 
therefore  is  a  general  description  not  limited  just  to  harmonic  behavior. 

In  the  instance  in  which  the  transfer  function  of  the  response  is  represented  by 


F(s)  =  Q-Ti)0-y2)...(-y-r„-i). 
O  -  a  ,)(s  -  a  2)...(s  -  a  „) 


(2) 


where 

Y  i  ~  a  zero  of  the  transfer  function,  and 


a  i  =  a  pole  or  singularity  of  the  transfer  function, 
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the  Laplace  transform  of  the  transfer  function  yields  equation  (1).  A  partial  fraction 
expansion  of  equation  (2)  yields 


where  Aj  is  defined  as  the  residue  for  the  j"1  pole  a r  Performing  an  inverse  Laplace 

transform  of  equation  (3)  produces  equation  (1).  Thus  the  complex  natural  frequencies 
correspond  to  poles,  and  the  amplitudes  to  residues.  (Lager  et  al.,  1977). 

Returning  to  equation  (1),  suppose  that  a  change  in  variables  has  been  introduced  such 
that  fuk  =  eat .  Then  if  equation  (1)  is  an  equality  for  all  values  of  t,  the  equations 

A,  +  A2+---+A „  =  /„ 

AM\  +  A2/u2+---+An/un  =/, 

a\M]  +  A2/j.22  +•  •  -+An/ul  -  f2  ,  (4) 

A,Mr+A2iuN2-'+:.+AnM:-'=fN-> 

are  necessarily  satisfied,  and  equation  (1)  may  be  based  on  the  results  of  satisfying  these 
equations  as  close  as  is  possible  (Hildebrand,  1956).  The  /u ’s  are  also  to  be  determined, 
requiring  at  least  2«  equations,  and  the  equations  are  nonlinear  in  the  pi ’s  This  difficulty 
can  be  minimized  by  adopting  a  technique  known  as  Prony's  method  (Prony,  1795).  The 
following  description  of  Prony's  method  follows  that  given  by  Hildebrand  (1956). 

Let  be  the  roots  of  the  algebraic  equation 

A"  ~  A"-’  -  a 2 M"'2 - «„-,A  -  a„  =  0  (5) 

so  that  the  left-hand  side  of  equation  (5)  is  identified  with  the  product 
(A  -  A i )( A  ~  Mi Mn)  •  Proceed  to  modify  the  equations  in  (4)  by  multiplying  the 
first  equation  by  an,  the  second  by  a the  nlh  equation  by  a,,  and  the  (n  +  l),h 
equation  by  -1,  then  add  the  results.  Using  the  fact  that  each  ju  satisfies  (5),  the  result 
of  the  addition  is  in  the  form 


/«  - «„/o=0.  (6) 

A  set  of  N  -  (n  - 1)  additional  equations  with  a  similar  form  can  be  obtained  in  the  same 
manner  by  beginning  instead,  successfully,  with  the  second,  third,  fourth, ■■■,(N-n)lh 
equations.  Thus,  it  is  seen  that  (4)  and  (5)  imply  the  set  of  N  -  n  linear  equations 
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f„-\a\  + /»-2«2+- ••/<>«»  =/» 

/„«!  +  fn-\a2+"'f\an  =  fn+ 1 

(7) 


f N-2a I  +  f N-2a2+'"  f N-n-\a „  ~  f N-\. 

The  ordinates  /*  are  known,  which  means  this  set  generally  can  be  solved  directly  for 
the  na  's  if  N  =  2 n.  Once  the  a 's  are  determined,  the  n/u 's  are  found  as  the  roots  of  (5) 
and  may  be  real  or  complex.  The  equations  in  (4)  then  become  linear  equations  in  the 
nA 's,  with  known  coefficients.  The  A 's  can  be  determined  from  the  first  n  of  these 
equations.  Note  that  the  non-linearity  of  the  system  has  been  concentrated  in  the  single 
algebraic  equation  given  in  (5). 

The  outline  just  given  provides  a  summary  of  the  mathematical  essentials  of  the 
waveform  decomposition  methodology.  The  decomposition  output  comes  directly  from 
an  exact  solution  to  the  specified  number  of  equations  describing  the  input  data. 
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